Learning Image-Derived Eye Movement Patterns to Characterize Perceptual Expertise

Experts have remarkable capability of locating, identifying and categorizing objects in their domain-specific images. Eliciting experts’ visual strategies will benefit image understanding by transferring human domain knowledge into image-based computational procedures. In this paper, an experiment conducted to collect both eye movement and verbal description data from three groups of subjects with different medical training levels (eleven board-certified dermatologists, four dermatologists in training and thirteen novices) while they were examining and describing 42 photographic dermatological images. We present a hierarchical probabilistic framework to discover the stereotypical and idiosyncratic viewing behaviors exhibited within each group when they are diagnosing medical images. Furthermore, experts’ annotations of thought units on the transcribed verbal descriptions are time-aligned with discovered eye movement patterns to interpret their semantic meanings. By mapping eye movement patterns to thought units, we uncover the manner in which these subjects alternated their behaviors over the course of inspection and how these experts parse the images.